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The research in this report was concerned with a student 
tutorial method of instruction and the effects of the. distribution 
(spacing or massing) of explanation of incorrect student responses 
on cognitive and .ittitudinal variables. The experimental design was 
a 3 x 2 x 3 factorial with seven observations per cell. Three types 
of programmed instruction treatment materials were employed reflecting 
the differential placement of the h.igh-difficulty frames. Version 1 
consisted of twenty-one difficult "integration" frames being placed 
every eighth or ninth frame. Version 2 was developed with these 
frames massed together at the conclusion of Units 1 and 2 , ten per unit. 
Version 3 consisted of experimental frames being presented at the end 
of Unit 2* 

Therefore, two types of learning materials differing only in 
difficulty were included in the investigation. 

ii 




Two forms of feedback were employed, for the high-difficulty 
frames, either explanation or simply knowledge of results (KR) . An 
explanation consisted of a detailed statement concerning the particu- 
lar frame's content. 

Within the paired grouping of students, two roles were 
possible, either tutor-role or pupil-role. The third experimental 
role consisted of students studying individually. 

The results of this investigation showed that paired students 
(tutor-pupil) perform as well as the individuals for the difficult 
learning material while the tutor-role students performed the poorest 
on the conventional linear text when compared to the individuals. 

The coalition of student tutor and explanation did not aid 
the learning of the difficult learning material, although explanation 
itself was found to reduce student errors for those students in the 
Version 3 treatment condition. 

Student attitudes were not adversely affected by the mani- 
pulation of errors via the "integration" frames (error frames) . 

The magniture of positive attitudes appeared to be dependent upon 
criterion test performance. Initial student attitudes toward modern 
instructional technology were not found to significantly effect 
performance . 
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INTRODUCTION 



The research in this report was concerned with a student tutorial 
method of instruction and the effects of the distribution (spacing or 

I 

massing) of explanation of incorre t stu- ant responses on cognitive and 
attitudinal variables. Interest in a tutorial method of instruction has 
become a major concern of educational psychologies since the advent cf. 
Programmed Instruction (pi) and Computer-Assisted Instruction (CAI) . 

It may well be that a major use of student resources has beer- 
overlooked with the inception of these new technological innovations. 

Yancey (1968) states it this way: 

-■ What might one do until the computer comes? Perhaps in the rush 
toward esoteric hardware for deliverance from our educational woes, 
we have overlooked a method of instrumentation that is quite sophisti- 
cated itself. This piece of gear is indeed a very good general purpose 
computer. It is readily available to any researcher due to the fact 
that it is produced happily by unskilled labor, resulting in (JciJ.f.6 5in 
excess of supply and demand. It requires almost no maintenance and 
may be rented cheaply. It can be programmed via oral instructions 
spoken in ordinary English and/or via written material. It can also 
accept and process oral or written output, even.jiiagrams drawn on the 
blackboard. Its output can even be a spoken word or a drawing on a 
surface. It can even project slides and point to relevant areas while 
giving concurrent verbal interaction. This computer has many names, 
but may be classified under the generic label "Student" (pp. 20-21). 



Paired Literature 

There have been far fewer studies on this topic than one would 
generally suppose, and thus the generalizations in the literature-- that, 
pairing is a*' advantageous method of programmed learning because it is 
more economical, is less boring for students, and that retention is better 
for paired students — are generalizations not well supported. There is, 
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however, limited evidence that programmed learning in pairs may at least 
be as effective as individual programmed learning. Consequently, the 
question arises as to which is the best method of pairing. 

In the experiments to date, three main methods have generally 
been employed: students have been paired at random, or in pairs of 

similar ability or pre-knowledge (homogeneous) , or in pairs of dissimilar 
ability (heterogeneous) . Sawairis (1966) paired his Ss on the basis of 
both similar ability and dissimilar ability in the subject matter 
(geometry) as shown by pretest scores. No significant differences in 
posttest performance between the experimental groups was found. Hartley 
and Cook (1967) carried out eight miniature experiments on homogeneous 
and heterogeneous pairing. These authors found no evidence to support 
the hypothesis that working in a heterogeneous pair is an advantage for 
the poorer member. High-ability Ss paired with low-ability Ss were not 
hindered by the lower-ability member. 

Grubb (1964) paired thirty Ss into high-ability pairs and low- 
ability pairs on the basis of their scores on the College Entrance 
Examination Board (verbal only) . His results indicated that Ss paired 
according to verbal ability perform as well in statistics as their 
controls (individuals). No significant time difference was found. 

Austwick (1965) compared three types of pairing with twelve- 
year-old Ss. In this experiment high-ability Ss were paired together, 
low-ability Ss were paired together, and a random pairing was used. 

In addition there was a control group in which Ss worked individually 
on the same program. Type of pairing was not found to significantly effect 
posttest performance. The paired groups performed as well as the indivi- 
duals, but completed the one hundred fifty frame program in less time. 
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Dick (1963) was concerned with the effect upon retention of 
active participation in the form of discussion. The purpose of Experi- 
ment 1 was to determine if paired study (random pairs) of a 3,500 frame 
modern algebra course would result in superior retention in comparison 
to individual study of the program materials. The immediate posttest 
scores indicated no significant differences between paired and individual 
Ss. The paired group required significantly longer to complete the 
program; an average of 3.7 minutes per unit or less than two seconds 
longer per frame. The author reports that there were no attitudinal 
diff ere nces between the experimental groups . 

Experiment 11 dealt with re-test results after a period of one 
year. £s were the same participants involved in experiment 1. Eighty 
percent of the original number of Ss were available for re-testing. 

After adjusting for initial performance, paired students' scores were 
found to be significantly higher than individuals. The best single 
predictor of retention was the final examination score. 

Two studies have reported results which do not support the use 
of a type of pairing within a PI framework. Jones (1963) reported on 
the use of intrinsic programs presented via an Autotutor to teach 
managerial concepts to groups of three apprentices. Group presentation 
was less effective than individual presentation. Noble (1967) also 
found individual study superior to paired study of PI. 

Besides ability- type pairings, personality variables have 
been used as a basis for student pairing. Dick and Sequin (1963) 
investigated the role of a dominance-submission factor on the performance 
of paired Ss. This personality variable did not predict the academic 
performance of the pairs. Recent work conducted by Majer (1969) has 
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shown that individual personality, ability, and background characte- 
ristics differentially predict performance success. 

Tutorial Studies 

None of these studies attempted to differentiate student roles 
during the learning of the PI materials. Ss essentially worked indepen- 
dently and were only involved in verbal interaction with their respective 
partners when difficult concepts were encountered. Two studies, however, 
did attempt to make a teacher-pupil role distinction within any given 
pair of Ss . 

Myers, Travers, and Sanford (1965) examined the effect of rein- 
forcement on verbal learning by pairs and individuals. The task consisted 
of sixty German words and their English equivalents presented on flash 
cards. This experimental condition is not exactly PI, but the situations 
are analogous. Four learning conditions were employed: (1) a student 

playing the teacher-role, (2) a student playing the pupil- role, (3) 
students who would switch roles at the midpoint of the task, and 
(4) students who worked individually. Ss received feedback by reading 
the feedback statement on the reverse side of the card. Feedback was 
always in the form, "the right answer i s . " Condition 2 Ss 

(pupil-role only) showed superior learning to all other treatments . 
Condition 3 (reverse roles) performed better than Condition 1 (teacher- 
role only) . No significant differences were found between Conditions 
3 and 4. 

From the results of the above study, it appears that students 
can be paired to each other effectively with materials required 
rote learning, although the teacher-role Ss showed the poorest 
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performance. It may be that students cannot adequately perform when 
given added responsibilities. Still, the nature of verbal reinforcement 
given by another person is important for meaningful learning. 

Another study which attempted to simulate a tutorial relation- 
ship was one conducted by Yancey (1968). He investigated the 
feasibility of using CAI programming techniques (without hardware) as a 
supplementary teaching aid for college students. An introductory 
psychology class was divided into three matched groups. Two groups used 
CAl-type programs with two Ss working together — one jS playing the 
tutor role and the other j> serving as the typical student. One CAI 
group used a set of materials prepared to motivate the students to relate 
to the principles of psychology in addition to a PI text. The second 
CAI group used a PI text for the initial five weeks. During the 
succeeding five weeks the tutor-role Ss were given copies of an intro- 
ductory psychology text. Their task was to develop short questions 
based upon the chapters and then to pose these questions to students. 

Ss were allowed to change roles during the course of the experiment. 

It was assumed that all Ss had read the assigned chapters before the 
weekly experimental sessions. The third group merely participated in 
extra-class psychological experiments. 

The two CAI-like groups performed somewhat better than the 
group which simply participated in order psychological experiments. 
Although the differences were not significant with regard to cognitive 
variables, the CAI groups rated the experience as being more meaningful. 
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Low and High Difficulty Learning Materials 

In order to study the effects of a tutorial learning situation, 
special materials are required. A problem arises as to how to induce 
tutoring when a programmed text designed for almost total correct 
responses (low error rate) is used. In this case the tutor is virtually 
unnecessary in that his function would mainly be the imparting of know- 
ledge of results. One method to induce a tutorial relationship would be 
to include a number of frames within the PI units which would result in 
errors on the part of the student. In this way the tutor's role would 
be just that, a tutor attempting to explain any misconceptions on the 
part of the student. 

It is contended that the effects, if any, of a tutor can only 
be shown by the use of difficult learning material. Most of the earlier 
studies employed conventional linear programs which were written for 
error-free responding, the notion being that Ss only learn from correct 
responses. When paired students are together, there is really no 
reason to assume that greater learning should result. Through the use 
of difficult learning materials in conjunction with a conventional PI 
text, this hypothesis can be explored. 

With the addition of the difficult "integration" frames (to be 
described later), two other independent variables can be manipulated; 
the spacing or distribution of the error frames and the detail to which 
the tutor-role £ explains the incorrect answer. We shall deal with the 
latter of these two independent variables first. 
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Explanation of Incorrect Answers 
One should possibly view explanation of incorrect answers as 
really being a form of repetition. Within the realm of PI repetition 
has not been found to be a significant factor for increasing learning 
and retention of meaningful verbal discourse material (Reynolds & 
Glaser, 1964), although repetition has been found to be a key factor in 
studies of overlearning (Postman, 1961) and retroactive inhibition 
(Briggs, 1957) with paired associate learning, it may well be that the 
ineffectiveness of repetition is due to the relatively low difficulty of 
the frames initially, wi.h repetition being virtually unnecessary. 

When repetition is focused upon errors, the results are some- 
what different. Holland and Porter (1961) focus upon the effects of 
repetition of incorrectly answered items within a PI framework. Using 
the Holland and Skinner program. The Analysis of Behavior , error rates 
were varied on one section of a ten unit text. One experimental group 
used the entire program as written by the original authors, repeating 
missed items at the conclusion of every item set until the item was 
answered incorrectly. The other group did not use this review feature. 
Instead, each frame was answered only once, regardless of the correct- 
ness of the response. At all three-item difficulty levels, the non- 
review group showed lower performance than the review group. The 
results of a six-month retention test revealed the same findings. 

Explanation as a variable has been investigated by Bryan and 
Rigney (1956). The Pull-Tab was a device in which the subject received 
not only a "right" or "wrong" indication after his choice, but also a 
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somewhat detailed explanation of "why" a response was incorrect. The 
data illustrated that the combination of immediate knowledge of results 
(KR) plus explanation if the student was in error, produced significantly 
higher cirterion test scores than if no explanation had been given. 

This finding is consistent with the result of Krumboltz and Bonawicz's 
(1962) study which showed that feedback presented in the context of a 
complete sentence was superior to the presentation of the feedback term 
alone. 

The importance of Bryan and Rigney's research from a historical 
point of view is that it investigated immediate knowledge of results as 
a factor existing on a continuum with varying degrees of effect. 
Explanation and/or repetition does seem to have an effect with difficult 
learning material. 



Spacing of the Difficult Material 
The spacing or distribution of difficult material has not been 
investigated with regard to its effects in conjunction with explanation 
or KR on learning and student attitudes. A major concern involved in 
the present study was an attempt to determine whether explanation would 
be more effective when given periodically as in the Bryan and Rigney 
(1956) study, or in a semi-massed session at the end of each of the two 
PI units (Holland & Porter, 1961), or in a massed fashion at the 
conclusion of the second PI unit. With these three different spacings, 
it is possible to assess the relative effectiveness of the coalition 
of a tutor plus explanation or KR of incorrect answers in three, 
different distinct learning situations. 
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Attitudinal Effects of Errors 

Variations in the spacing of the difficult material also allowed 
for the study of the effects of error spacing on student attitudes. 

The effects of errors upon student attitudes is fairly well documented. 
Wodtke (1966) reported that a student's attitudes toward a certain 
method of instruction might be a result of his performance. Students 
who performed better on CAI tasks showed more favorable attitudes after- 
wards than those who performed poorly. Mathis, Smith, and Hansen (1968) 
found that Ss who studied CAI material with which they were familiar 
showed more positive attitudes than those who studied unfamiliar 
materials. Ss who committed the least number of errors were also 
found to le more favorable to the experience. Eigen and Feldhusen (1964) 
reported that student attitudes toward a program are generally negatively 
correlated with the amount of self-identified errors made while taking 
the program. 

The results of these investigations show evidence that 
attitudinal measures are correlated with performance measures— the 
more errors involved, the less favorable student reaction tended to be. 

Hypotheses 

Based upon the reviewed studies and theory, the following 
hypotheses were formulated for the experiment. For conceptual clarity 
and explanatory insight, the research hypotheses were categorized into 
the following groups: (1) tutorial role, (2) explanation or KR of 

incorrect answers, and (3) distribution of incorrect answers. 

I 
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Tutorial Role 

For the conventional linear text material, no effects of a tutor 
should be encountered. In essence, the tutor merely monitor's the 
student's progress through the materials. 

For the difficult material ("integration" frames), it is 
hypothesized that the coalition of tutor and explanation of incorrect 
answers should lead to increased learning of this material for both 
tutor and pupil over that of the individual study plus explanation. 

Explanation of Incorrect Answers 

A dual role of explanation is hypothesized. Explanation 
should increase the learning of the difficult material, and as such, 
result in an increase in the positiveness of student attitudes. The 
second role of explanation is predicated on the basis of the known 
negative correlation between number of errors and student attitudes. 

Spacing of the Difficult Material 

An exploration into the relative effectiveness of explanation 
or KR of incorrect answers in three different situations in combination 
with a tutor is being made. In this regard, no predictions can be made 
based upon other previous work or theoretical position. 

The distribution of errors (difficult material) should have an 
adverse effect upon student attitudes. The larger numer of errors 
committed in succession (massed condition) will have a more adverse 
effect upon student attitudes when compared to a distributed error 
condition (every eighth or ninth frame) or a semi-massed error condition. 
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Exploratory Analyses 

The relationship between performance and initial student 
attitudes to modern instructional technology will be studied. Also to 
be explored is the relationship between Computer-Based Testing attitudes 
and test performance. Finally, the use of confidence ratings in the 
determination of test internal consistency reliablity will be explored. 

All hypotheses will be tested at the .05 alpha level. 



METHOD 



Subjects 

One hundred twenty-six students (90 females and 36 males) were 
obtained from an introductory educational psychology class at The Florida 
State University during the Winter Quarter, 1969. Ss were randomly 
assigned to the treatment groups. Five Ss were eliminated from the 
investigation; three Ss had dropped the course from their schedules 
while two Ss completed the first treatment session but did not complete 
the entire treatment procedure due to illness. Thus, the total N was 
reduced to 121. Participation in the investigation was a course require- 
ment with the Ss' scores on Test 1 partially determining their final 
educational psychology course grade. 

Experimental Design 

The experimental design of the present investigation was 
essentially a 3 x 2 x 3 factorial with seven observations per cell. Due 
to certain unavoidable problems three experimental cells did not have an 
n of seven. Table 1 presents a description of the design. Three types 
of treatment materials with differential spacing of errors were used. 

For each set of materials two forms of feedback for incorrect answers 
were possible, either explanation or knowledge of results (KR) . The 
third independent variable was subject role (teacher-role, pupil- 
role, or individual). 
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TABLE 1 

EXPERIMENTAL DESIGN 







Type of Feedback for 




Student 


Roles 


Treatment Materials 


Incorrect Answers 


Teacher 

(N=40) 


Pupil 

(N=40) 


Individual 

(N=41) 


Version 1 




Explanation 


n=7 


n=7 


n=7 


Spaced "Integration" 
Frames 


Knowledge of Results 


n=7 


n«=7 


n=6 


Version 2 




Explanation 


n=7 


n=7 


n=7 


Semi-Massed 

"Integration" 


Frames 


Knowledge of Results 


n=5 


n=5 


n=7 


Version 3 




Explanation 


n=7 


n=7 


n=7 


Massed 

"Integration" 


Frames 


Knowledge of Results 


n=7 


il 

a 


n=7 



Treatment Materials 

Low difficulty materials .--Two programmed units from Gibson's 
Educational Psy chology were used as the treatment materials. Topics 
appropriate for an introductory educational psychology class, intelli- 
gence testing (Unit 1) and Piaget's theory of cognitive development 
(Unit 2) were used. The text is basically a multiple-choice program, 
written in a linear format with approximately one hundred frames per 
unit. A typical program frame is depicted in Figure 1. This small-step 
program required an active, overt response on the part of the student for 
every frame. The average student error rate per unit, as reported by 
Gibson (1967) was less than 3' percent. 



Frame Number 41, Unit 1 

MA 

The formula for determining the ratio IQ is: 10 * CA x 100. When we 

determine the ratio IQ, we are taking into consideration: 

a. mental age 

b. number of correct answers to questions 

c . chronological age 

d. all of these 

41. 

Figure 1. — Example of Program Frame from Gibson (1967) 

High difficulty materials ♦- -A number of "integration".:frames were 
designed to evoke student errors during the treatment sessions. In most 
cases, these frames required the student to integrate, synthesise,, and 
apply concepts developed in Units 1 and 2. Twenty "integration" frames 
were initially developed by the investigator for both Units 1 and 2. 

A pilot study was conducted to determine the empirical -difficulty of thA 
"integration" frames. 



THE PILOT STUDY 



Subjects 

Twenty £s obtained from the introductory psychology classes at the 
Florida State University were used in the pilot study. Participation in 
this experiment was considered a requirement for the Ss' psychology class. 

M aterials 

Two programmed units from Gibson 1 s (1967) Educational Psychology 
were used in original form with the addition of twenty "integration" 
frames placed at the conclusion of each unit. 

Procedure 

During the Fall Quarter, 1968, Ss studied Units 1 and 2 indivi- 
dually in a quiet room. Ss were told that they were aiding in the 
development of these materials for future educational psychology classes. 
Ten Ss studied Unit 1 and ten Ss studied Unit 2. Ss were required to 
make all overt responses to the program frames. 

Ss were required to rate certain frames according to difficulty 
using a five point Likert- type scale ranging from extremely easy (1) to 
extremely difficulty (5) . All twenty "integration" frames for both 
units were rated in terms of difficulty in addition to twenty Gibson 
(1967) frames chosen at random from each unit. 

Results 

Of the forty "integration" frames, twenty-one (eleven for 
Unit 1 and ten for Unit 2) were selected for use in the investigation. 
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The rejected frames showed error rates ranging from 20 percent to 75 
percent. The twenty-one frames selected had an error rate of 90 
percent or better for the pilot Ss . 

A one-way analysis of variance was computed for the ratings 
on the Gibson (1967) frames (Units 1 and 2) and the ratings for the 
"integration" frames. The results of this analysis revealed an JF 
value of 33.57, 1 and 18 d.f, £<.01. The mean difficulty rating for 
Gibson's (1967) frames was 1.80. The "integration" frame mean diffi- 
culty rating was 3.67. 



Discussio n 

The "integration" frames selected for the major investigation 
were indeed difficult as evidenced by the 90 percent error rate shown 
by these pilot Sjs . Further evidence, of difficulty, although subjective 
in nature, was shown by the significant difference in mean difficulty 
ratings between the original program frames and the "integration 1 .' 
frames . 



Development of the Three Versions of 
Treatment Materials 

With these "integration" frames three different versions of 
the treatment materials were developed with differences in the spacing 
of these frames. Version 1 consisted of these "integration" frames 
being spaced throughout Units 1 and 2 every eighth or ninth frame 
(Unit 1- -Frames 14, 27, 39, 48, 70, 83, 90, 95, 102, 111; Unit 2— 
Frames 9, 30, 39, 53, 61, 73, 77, 84, 88, 91). Version 2 was 
developed with the "integration" frames massed together at the 
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conclusion of the appropriate unit (Unit 1— Frames 115-125; Unit 2— 
Frames 91-100). Version 3 consisted of all the "integration" frames 
being presented at the end of Unit 2 (Frames 90-110). 

Explanations were developed for each of the "integration" 
frames as well as for the original frames. Figure 2 shows an 
'integration" frame along with its explanation. Each explanation 
consisted of a statement of the correct answer and a n um ber of 
statements designed to explicate the desired answers. 



Student sees this frame only: 



Given the following data: 



Tom 

Bill 

Mary 

Sue 

Jack 



Test A Test B Test C Test D 



10 


20 


10 


60 


20 


30 


10 


70 


10 


20 


10 


30 


40 


50 


10 


90 


50 


60 


10 


99 



a. The correlation between Test A and Test B is about 

b. The correlation between Test C and Test D is about 

a. about +1.00 

b. about 0.00 (zero) 

Teacher has frame plus the following explanation: 



The correlation between Test A and Test B is about +1.00. From 
a person a score on Test A, one can accurately predict his score on the 
second test (B) . Remeber: correlation expresses the extent to which 

scores on one test "go" with scores on another. 

The correlation between Test C and Test D is about 0.00 (zero). 
It would obviously be impossible, within such a group, to predict an 
individual s score on Test D just knowing his Test C score. The scores 
on Test D reflect an increase. These scores do not "go" together, 
therefore, one should expect an extremely low correlation be tween* them. 



Figure 2.— Example of a unit 2 "integration" frame complete 
with explanation. 
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Dependent Measures 



Pretest (Appendix A) 

For use as a pretest measure, twenty representative items were 
selected from the fifty item posttest (Test 1); ten items per unit. 

Attitude Scale Toward Instructional 
Media (Appendix B) 

A ten item Likert-type scale was devised to assess an 
orientational atcitudinal set toward modern instructional technology 
in general. The items allowed for responses on a five point scale 
ranging from Strongly Disagree to Strongly Agree. This scale is an 
adaptation of the Brown (1966) DAI attit.udinal measure. 

Test 1 (Appendix C) 

Test 1 consisted of fifty multiple-choice items principally 

chosen from a number of introductory educational psychology textbooks 

with a minimal number (7) being devised by the experimenter. This 

test was used to measure student learning of the original Gibson 

1 

(1967) programmed units. Twenty-five items were devoted to both 
Units 1 and 2. 

Test 1 was administered via the IBM 1500 instructional system. 
This system consists of a central processing unit, storage units, a 
transmission control unit, and twenty-one cathode ray tube terminals 
(CRT) . Student input for all responses was restricted entirely no 
light pen responding. During the administration of Test 1, Ss were 
required to rate their confidence in the correctness of their answers 
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via a seven-point Likert- type scale. This scale ranges from 1 
(unsure) to 7 (positive my answer was correct). 

Two separate scoring procedures were employed to determine 
the internal reliability of Test 1. The coefficient of reliability 
was determined through the use of the analysis of variance. It is 
exactly equivalent to the Kuder-Richardson 20 estimate (Guilford, 

1954, p. 383). 

Method 1 consisted of assigning l's and 0's for correct and 
incorrect responses respectively. Method 2 scored each item as the 
product of "correctness (+1 or -1) times the confidence value for 
the particular item." For example, if student 1 answered item 1 
correctly and indicated a confidence rating of 7, his score would 
become +7 for that item. If he answered incorrectly and gave a confi- 
dence rating of 7, his score would be -7. The constant value of 7 was 
added to all transformed scores in order to eliminate the negative 
values obtained • 

Table. 2 presents the reliability estimates for Test 1 based 
upon the two scoring methods mentioned. Estimates for the entire 
fifty-item test are presented along with the estimates for the two 
units of the test. The scoring technique of obtaining the product 
of correctness times confidence increased the overall reliability from 
.645 to .687, which conforms to the conclusions drawn by Massengill 
and Shuford (1967). As to be expected, the reliability of the two 
units were increased with this n^^ procedure. 



t 
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KUDER-RICHARDSON 20 RELIABILITY COEFFICIENT FOR 
SCORING METHODS 1 AND 2 



Method 1 Method 2 

Reliability Reliability 



Test 


1 


(50 items) 


.645 


.687 


Part 


1 


(25 items) 


.546 


.578 


Part 


2 


(25 items) 


.404 


.451 



Programmed Instruction Attitude 
Scale (Appendix D) 

To assess student attitudes toward the PI experience, a modified 
version of the Brown and (1966) CAI attitudinal scale was used* Brown 
(1966) reported a reliability of .89 for the forty- item scale. The 
modified version (25 items) has been shown to have a Kuder-Richardson 
20 reliability of .822 (Smith, Hansen, & Hedle, 1968). 

Computer-Based Testing Attitude 
Scale (Appendix E~) 

Ten attitude items were selected from an adapted version of 
the Brown (1966) scale by Hansen and Schwarz (1968) to measure student 
attitudes toward the Computer-Based Testing (Test 1) used in the 
present investigation. 

Test 2 

Basically, Test 2 was a second administration of the twenty- 
one "integration" frames studied in Units 1 and 2. This constructed- 
response measure was administered via conventional paper and pencil 




21 



techniques. The purpose of this measure was to assess the effects of 
explanation upon the learning of difficult meaningful verbal discourse 
material. 






i 
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PROCEDURE 

Table 3 presents the experimental time schedule of the 
investigation. 

TABLE 3 





COMPLETION OF EXPERIMENTAL TEST 


SCHEDULE 






BY DAY AND NUMBER OF 


SUBJECTS 


(n = 121) 






Sequence of Events 




Day 








1 


5 7 


12 


1. 


General Instructions 


X 






2. 


Part 1, Pretest 


X 






3. 


Administration of Orientation 
Attitude Scale 




X 




4. 


Study of Unit 1 




X X 




5. 


Part 2, Pretest 




X 




6 . 


Study of Unit 2 




X 


X 


7. 


Administration of PI Attitude 
Scale 




X 


X 


8. 


Administration of Test 1 




X 


X 


9. 


Administration of Computer-Based 
Testing Attitude Scale 




X 


X 


10. 


Administration of Test 2 




X 


X 






n=121 


n=121 n=58 


n=63 




During the scheduled class period prior to the initiation 


of 


the 


treatment procedures (Day 1) Ss were given general instructions 



concerning the investigation with regard to their assignment to the 
various experimental groups and also their room assignments. Ss were 
only told that varying study conditions would be employed. At this 
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time Part 1 of the pretest was administered via conventional paper and 

pencil techniques. This pretest dealt with the concepts contained in 
Unit 1. 

Five days later the students reported to their assigned rooms 
at the regularly scheduled class meeting time. 

Each major treatment group (paired groups with explanation of 
incorrect answers, paired groups with KR of incorrect answers, and 
individuals) reported to a separate room. 

Copies of student directions for all treatments are presented 
in Appendix F. 

After the distribution the treatment materials (Unit 1 only 
at this time) jSs were instructed to read the instructions carefully 
and to pose any questions before study began. Following the instruc- 
tions, the attitude scale toward instructional media was completed. 
Study via the different modes then ensued for approximately one hour. 
At the conclusion of the class period, Ss were required to hand all 
treatment materials to the assistants before leaving. 

Two days later, Ss reported to their assigned rooms at the 
scheduled hour and received their materials from the assistants (Unit 
1 and 2). The assistants instructed the £s to complete study of Unit 1 
before, commencing to work on Unit 2. Upon completion of Unit 1, Ss 
completed Part 2 of the pretest. After completing this test (paper 
and pencil), Ss began study of Unit 2. Fifty-eight Ss finished the 
treatment materials during this two-hour session. Ss who completed 
the materials during this session were tested that afternoon and 
evening at the Florida State University CAI Center. Approximately 10 
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Ss were, tested at any one of the scheduled testing times. Ss who did 
not finish at this time handed in their materials to the assistants al. 
the conclusion of the class period. 

On the twelfth day, the 63 Ss who had not completed Unit 2 
reported to the main classroom to conclude study. All Ss completed 
the Pi attitude questionnaire immediately upon conclusion of study, 
regardless of the session in which they had completed the. treatment 
materials . 

Testing was accomplished as follows. Ss reported to the 
Florida State University CAI Center at their appointed time. After 
general instructions concerning terminal operations, Test 1 was 
administered via the IBM 1500 instructional system during the after- 
noons and evenings of February 27, 1969, and March 1, 1969. Ss were 
required to rate their confidence in the correctness of their answers 
for every Test 1 question. _Ss were not given KR knowledge of results 
about their Test 1 responses, and were forced to rate each answer 
before continuing to the next question. At the conclusion of 
Test 1, three scores were displayed to the Ss on the CRT screen; 

(1) the score for Unit 1, (2) the score for Unit 2, and (3) the total 
score for Test 1. 

Immediately following the administration of Test 1, the 
Computer-Based Testing Attitude Scale was completed by the. Ss . Test 
2 was then administered to Ss in a quiet room at the CAI Center. 

Total testing time per subject was one-hour. Two Ss were 
tested later in the. week due to scheduling problems. 




RESULTS 






There were one hundred twenty»one student performance records 
available for t.he analyses. 

For efficiency of comparison, five mean scores were inserted 
into a number of analyses with the error term being adjusted 
(Kempthorne, 19.52, p. 174) by a loss of 1 df for each inserted score. 

For the present investigation, all independent variables were 
assumed to be fixed factors. 

Pretest-Posttest Results 

To determine whether or not students learned during the 
experimental sessions a2x2x3x2 analysis of variance for three 
types of treatment materials, two forms of feedback, and three student 
roles, and pretest and posttest Test 1 scores (repeated measure) with 
seven replications was calculated for the Test 1 scores. Since these 
two dependent measures were not comparable with regards to the same 
total number of .items, all scores were converted to proportion 

correct before analysis. 

! 

The results of this analysis (Table 4) revealed that Ss showed 
significant learning of the low difficulty Gibson (1967) material 
(IP=486.0, 1 and 103 dT, <.001). A significant interaction between 
type of feedback (explanation or KR) and pre and posttest Test 1 
scores was found (F=7.82, 1 and 103 df, £. < .01) . On the pretest, 

Ss assigned to the KR groups scored lower than the Ss assigned to the 
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explanation groups while the reverse was true for the posttest. No 
other main effects or interactions were found to be significant. 



TABLE 4 

ANALYSIS OF VARIANCE OF PRE AND POSTTEST 
SCORES UTILIZING PROPORTION CORRECT 
AS THE DEPENDENT VARIABLE 



Source 


DF 


MS 


F 


Between Subiects 


Materials (I) 


2 


.0238 


2.48 


Feedback (J) 


1 


.0010 


— 


Roles (K) 


2 


.0223 


2.32 


IJ 


2 


.0157 


— 


IK 


4 


.0099 


— 


JK 


2 


.0120 


— 


IJK 


4 


.0132 


1.74 


Subjects within groups L(IJK) 


103 


.0096 




Within Subiects 


Tests (M) 


1 


3.6939 


486.04** 


IM 


2 


.0028 




JM 


1 


.0594 


7.82* 


KM 


2 


.0187 


2.46 


IJM 


2 


.0008 


— 


IKM 


4 


.0064 


— 


JKM 


2 


.0027 


— 


IJKM 


4 


.0071 




Mx Subjeccs within groups 


103 


.0076 





*p < .01 

**p <.001 



Test 1 Results (low difficulty material) 

A 3 x 2 x 3 analysis of variance for three types of treatment 
materials, two types of feedback, and three student roles with seven 
replications in each group was calculated for total posttest Test 1 
scores. The results of this analysis (Table 5) yielded a significant 
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difference on the role factor only (£*4.38, 2 and 103 df, p < .05). 
No significant effect for type c.f material or type of feedback was 
found. No interactions were found to be statistically significant. 

TABLE 5 

ANALYSIS OF VARIANCE FOR 3 TYPES OF MATERIALS, TWO 
TYPES OF FEEDBACK, AND 3 ROLE DIFFERENTIATIONS 
FOR TOTAL TEST 1 SCORES 



Source DF MS F 



Materials (I) 


2 


24.44 


r 


Feedback (J) 


1 


40.80 




Roles (K) 


2 


104.75 


4.38* 


IJ 


2 


11.78 




IK 


4 


2.36 


~ 


JK 


2 


19.51 




IJK 


4 


39.51 


— 


ERROR 


103 


23.91 





*p < .05 



To compare the mean scores for the three role variations, a 
Duncan’s Multiple Range Test was computed (Table 6). Individual role 
Ss showed higher performance when compared to the teacher-role Ss. 
Individual-role Ss did not differ significantly from the pupil-role 
only Ss, nor did the pupil-role only Ss differ significantly from 
the teacher-role. Ss. 
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TABLE 6 



DUNCAN'S MULTIPLE RANGE TEST FOR TOTAL 
TEST 1 SCORES FOR STUDENT ROLES 



Means 


Difference 
Between Means 


Least Significant 
Ranee Value 


Individuals 


- Teachers 






29.16 


26.05 


3.11 


2.23 * 


Individuals 


- Pupils 






29.16 


27.11 


2.05 


2.11 


Pupils 


- Teachers 






27.11 


26.05 


1.06 


2.11 


*p < .05 






Two 


similar analyses (3x2x3) were 


computed on the two 


units of Test 1 with the above mentioned role 


effect being the only 


significant 


finding in both analyses. A Duncan's Multiple Range Test 


(Table 7) was computed for 


the mean student role scores for Unit 1, 


Test 1, The 


initial result 


of the Duncan's Multiple Range Test was 


confirmed in this analysis. 










TABLE 7 






DUNCAN'S MULTIPLE RANGE TEST FOR TOTAL SCORE ON 




UNIT 1 OF 


TEST 1 FOR STUDENT 


ROLES 


Means 


Difference 


Least Significant 






Between Means 


Ranee Value 


Individuals 


- Teachers 






14.20 


12.45 


1.75 


1.38 * 


Individuals 


- Pupils 






14.20 


13.41 


.79 


1.31 


Pupils 


- Teachers 




• 


13.41 


12.45 


.96 


1.31 



♦p < .05 
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The results of Duncan's Multiple Range Test (Table 8) for 
the. mean scores for Unit 2, Test 1 revealed slightly different find" 
ings for Unit 2 than for Unit 1. Individuals showed significantly 
higher performance than either the tutor-role Ss (p <.05) or the 
pupil-role Ss (p<.05). Tutors did not differ significantly from the 
pupil-role Ss. 



TABLE 8 

DUNCAN ' S MULTIPLE RANGE TEST FOR UNIT 2 TOTAL 
SCORE OF TEST 1 FOR STUDENT ROLES 



Means 




Difference 
Between Means 


Least Significant 
Range Value 


Individuals - 


Teachers 






14.96 


13.61 


1.35 


1.23* 


Individuals - 


Pupils 






14.96 


13.70 


1.26 


1.17* 


Pupils 


leathers 






13.70 


13.61 


.09 


1.17 



*p < .05 

This performance difference cannot be attributed to time spent 



in learning the stimulus materials. A one-way analysis of variance for 
total time in minutes for tutors, pupil-role Ss, and individuals 
revealed no significant differences. 

Test 2 Results 

A 3 x 2 x 2 analysis of variance with seven replications was 
computed on the total number of errors on Test 2 (Table 10). Although 
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there were only twenty-one "integration" frames, a total of forty-one 
possible errors could be committed by the Ss . 



TABLif- 9 

ANALYSIS OF VARIANCE: TEST 2 TOTAL ERROR SCORES 



Source 


OF 


MS 


F 


Materials 


(I) 


2 


235.19 


7.30 ** 


Feedback 


(J) 


1 


.001 




Roles 


00 


2 


17.32 


— 


IJ 




2 


119.33 


3.70 * 


IK 




4 


75.07 


2.29 


JK 




2 


27.77 




UK 




4 


35.25 


— 


ERROR 




103 


32.74 





*p < .05 

**p < .01 



A significant material effect was found (F = 7.30, 2 and 
103 df, p < .01). The Ss who studied Version 3 of the materials 
committed fewer errors on Test 2 than the Ss who had Versions 1 and 
2. No significant differences were found for either type of feedback 
or student role. The manner in which explanation was presented (via 
tutor or reading) was not found to significantly effect Test 2 per- 
formance. A significant second-order interaction (F = 3.70, 2 and 
103 jdf , p < ,05) was found between type of materials and type of 
feedback. A graphical representation of this interaction is presented 
in Figure 3. The cause of this effect appears upon first inspection 
to be due to the fact that the explanation £s showed fewer errors on 
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Version 3 (massed condition) than did the KR £s with the opposite 
effect being true for Versions 1 and 2 of the treatment materials. 




Special Semi-massed Massed 
errors errors errors 



VERSION OF TREATMENT MATERIALS 

Figure 3. — Mean number of errors for total test 2 scores for 
materials by feedback. 

A Duncan's Multiple Range Test was computed in order to 
isolate the cause of the interaction. The results of this analysis 
are. presented in Table 10.* As can be seen, no significant difference 
was found between mean error scores for the explanation £s who studied 



*Only the significant mean differences are reported. 
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TABLE 10 

DUNCAN'S MULTIPLE RANGE TEST FOR 
EXPLANATION BY FEEDBACK 



Means 




Mean 

Difference 


Least Significant 
Range Value 


Explanation (1) 
23.05 


- Explanation (3) 
15.45 


7.60 


3.94 * 


Explanation (2) 
21.57 


- Explanation (3) 
15.45 


6.12 


3.87 * 


KR (1) 
20.92 


- Explanation (3) 
15.45 


5.47 


3.78 * 


KR (2) 
19.80 


- Explanation (3) 
15 .45 


4.35 


3.66 * 


KR (3) 
19.33 


- Explanation (3) 
15.45 


3.88 


3.47 * 



*p < .05 



Versions 1 and 2 while both these means were significantly greater for 
the Ss who studied Version 3. No significant differences were found 
between Version 1 and 2 and 3 for the KR Ss. Explanation Ss utilizing 
Versions 1 and 2 did not differ significantly from the. KR Ss for the 
same, treatment materials. Explanation Ss showed significantly less 
errors for Version 3 than the KR Ss, this difference, being seen as 
the cause of the. interaction. 

The. fact that fewer errors were committed by Ss who used 
Version 3 is borne out also from the significant materials effect 
found in the initial. 3x2x3 analysis. Results of a Duncan's 
Multiple. Range. Test. (Table. 11) showed that the mean differences in 
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number of errors made on Test 2 by Ss using Versions 1 and 2 was not 
statistically significant while the Version 3 Ss made significantly 
fewer errors than either of the two other treatment groups. 

TABLE 11 



DUNCAN'S MULTIPLE RANGE TEST FOR MEAN NUMBER OF ERRORS 
FOR 3 VERSIONS OF MATERIALS FOR TEST 2 



Version 


Mean 

Difference 


LSR 


I 


3 






2l798 * 


17738 


4.60 


2.59* 


1 


2 






21.98 - 


20.69 


1.29 


2.46 


2 


3 






30.69 


17739 


3.30 


2.46* 



*p <..05 



This effect seemingly holds true for both sections of Test 2. 
Two analyses of variance were computed for the mean number of errors 
for both sections of Test 2. Part 1 results (Table 12) revealed a 
significant material effect. (F = 8.56, 2 and 103 df, p<.01), and a 
significant seccnd-order interaction between materials and feedback 
(F = 4.05, 2 and 103 df ? p<«05). This interaction is depicted in 
Figure 4. The effect seems to be of the same order as found in the 
overall Test 2 analysis. 
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TABLE 12 

ANALYSIS OF VARIANCE: PART 1, TEST 2 

ERROR SCORES 





Source 


DF 


MS 


V 





Materials (1) 


2 


145.74 


8.56 ** 


Feedback (J) 


1 


1.79 


— 


Roles (K) 


2 


39.88 


— 


IJ 


2 


69.02 


4.05 * 


IK 


4 


14.73 




JK 


2 


3.50 


— 


IJK 


4 


26.99 


— 


ERROR 


103 


17.03 





*p < .05 

< .01 



13 




Spaced Semi -massed Massed 

errors errors errors 



VERSION OF TREATMENT MATERIALS 

Figure 4. — Mean number of errors for part 1 of test 2 for 
materials by feedback. 
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Results of the analysis of variance for Part 2, Test 2 
(Table 13) showed nearly the same findings. None of the main effects 

TABLE 13 

ANALYSIS OF VARIANCE: PART 2, TEST 2 

ERROR SCORES 



Source 


DF 


MS 


F 


Materials (1) 


2 


9.60 




Feedback (J) 


1 


.39 


— 


Roles (K) 


2 


9.07 


— 


IJ 


2 


30.39 


4.42 * 


IK 


4 


4.31 


— 


JK 


2 


2.91 


— 


IJK 


4 


12.27 


- --- 


ERROR 


103 


6.88 





*p < .05 

was significant, but a significant interaction between materials and 
feedback was found again (F * 4.42, 2 and 103 d£, p < .05). This 
relationship is shown in Figure 5. Inspection of Figures 3, 4, and 5 
shows that nearly the same relationship seems to hold for both Parts 
1 and 2, Test 2. Explanation seemed only to have an effect when 
received in massed "integration" frame condition. 
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Figure 5. --Mean number of errors for part 2 of test 2 for 
materials by feedback. 



Programmed Instruction Attitude Questionnaire 
A 3 x 2 x 3 analysis of variance was calculated with the 
dependent measure being the total attitude scale score. No signifi- 
cant differences were found for any of the independent variables. In 
other words, manipulation of the spacing or massing of errors did not 
have an adverse effect upon student attitudes as measured by this 
attitude scale. 

Ss were then grouped into High and Low PI attitudes. The 
median attitude scale score was used as the basis for the division. 

Ss were then compared with regard to type of materials (spacing of 
errors) and student roles via the general linear hypothesis model 
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(Graybill, 1961). The dependent variable was the total Test 1 per- 
formance. The results of this 2x3x3 analysis of variance (See 
T-ble 14) revealed that the High PI attitude Ss shewed significantly 

TABLE 14 

ANALYSIS OF VARIANCE j HIGH-LOW PI ATTITUDES, 3 TYPES 
OF MATERIALS, AND THREE ROLE DIFFERENTIATIONS 
FOR TEST 1 SCORES 



L^urce DF MS 



Attitudes (I) 


1 


96.03 


4.33 * 


Materials (J) 


2 


25.06 


1.13 


Roles (K) 


2 


86.04 


3.88 * 


IJ 


2 


.77 




IK 


2 


20.37 


i rm ti rm 


JK 


4 


3.03 


M — _ 


UK 


4 


70.52 


3 . 18 ** 


ERROR 


103 


22.16 





*p < .05 

**p < .01 



higher Test 1 performance than the Low PI attitude Ss (F = 4.33, 1 
£nd 103 df, p < .05) . A significant role effect was again found 
(F = 3.84, 2 and 103 df , p < .05). The effect for spacing of errors 
(materials) was not significant. None of the second-order inter- 
actions reached statistical significance. However, a significant third 
order interaction between High and Low attitudes, types of materials, 
and roles was found (F = 3.18, 4 and 103 df, p < .05). This relation- 
ship is depicted in Figure 6 (high attitude Ss) and Figure 7 (low 
attitude Sis) . 



MEAN NUMBER CORRECT ON 
TEST 1 
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Figure 6. --Mean test 1 scores f:or high PI attitude students, 
materials and student roles. 
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Apparently, individuals who received spaced errors ("integra- 
tion" frames) performed well on Test 1, and consequently, showed a 
favorable reaction; whereas, individuals receiving massed errors at 
the conclusion of treatment showed poor FI attitudes, although these 
Ss performed well. Teachers who used the massed error material 
(Version 3) did well on Test 1 and also showed favorable PI attitudes. 
On the other hand, teachers who used the same materials and performed 
poorly on Test 1 did not favor PI. 

Computer-Based Testing Attitudes 
A 2 x 3 analysis of variance was computed for High-Low ■ 
Computer-Based testing Attitudes (division at - the median) and the 
three student roles. The dependent variable in question was Test 1 
performance. The results of this analysis are presented in Table 15. 

TABLE 15 

ANALYSIS OF VARIANCE: HIGH-LOW COMPUTER-BASED 

TESTING ATTITUDES AND THREE RULES 
FOR TEST 1 SCORES 







Source 


DF 


MS 


F 



Attitudes (J) 


1 


118.54 


5.46 * 


Roles (J) 


2 


103.46 


4.76 * 


IJ 


2 


80.37 


3.75 * 


ERROR 


115 


21.72 


..... 



*p < .05 
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High attitude Ss performed significantly better than Low altitude Ss 
(JF = 5.46, 1 and 115 df, p <.05). A significant role effect was again 
found (F = 4.76, 2 and 115 df, p .05), with individuals showing 
higher performance than teacher-role Ss or pupil-role Ss . A significant 
interaction was found (F = 3.75, 2 and 115 df, p < .05) between 
attitude and student role. This relationship is depicted in Figure 8. 



STUDENT ROLES 

Figure 8. Mean test 1 scores for high-low computer-based 
testing attitudes by three roles. 

High attitude Ss performed basically alike with the difference lying 
in the Low attitude Ss. Low attitude teacher-role Ss and pupils 
showed similar performance, whereas Low attitude individuals performed 
on a par with the High attitude pupils and individuals. 
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Confidence Ratings 

A 3 x 2 x 3 analysis of variance was calculated on the total 
confidence rating scores per subject obtained from the Test 1 
administration. No significant differences were found .:or any of the 
independent variables. 

Attitude Scale Toward Instructional Me d ia 
Student Orientation 

A 3 x 2 x 3 analysis of variance with seven replicat* . . s p.* 
cell was calculated on the total attitude score toward instructional 
media. No significant differences were found for any of the independent 
variables . It was concluded that the students in the various treatment 
groups did not differ in initial attitude. 

Ss were then grouped into High and Low "Orientation" attitude 
groups. The median "orientation" attitude was used as the basis for 
the division of the groups. A 2 x 3 analysis of variance was computed 
for High and Low "orientation" attitudes and the three types of student 
roles, the dependent variable being total Test 1 performance. The 
mean scores for the six groups are presented in Table 16. 

TABLE 16 

MEAN "ORIENTATION" ATTITUDE SCALE SCORE 





FOR 


THREE STUDENT 


ROLE 




"Orientation" 


Attitudes 




Student Roles 


71 -> 






Teachers 


Pupils 


Individuals 


High 




28.86 


28.33 


39.32 



Low 



28.11 



26.00 



29.00 
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The method of the general linear hypothesis (C-raybill, 1961) was used 
for the unbalanced data. 

High or Low "orientation" attitudes were not found to have a 
significant effect upon performance while a significant effect for 
student role was again found (F = 4.42, 2 and 115 df, p < .05) 

No significant interaction was found (See Table 17) . 






AN ^ S “ ° F VARIANCE: HIGH-LOW ORIENTATION ATTITUDES 

AND THREE ROLE DIFFERENTIATIONS FOR TEST 1 SCORES 




Attitudes (J) 
Roles (J) 

IJ 

ERROR 



1 

2 

2 

115 



64.63 

103.51 

10.85 

23.42 



2.76 
4.42 * 
.46 



*P < .05 



DISCUSSION 



This research should be examined with the following points 
in mind relating to the external validity of the study: (a) the Ss 

were not randomly selected, but participated in the investigation 
to fulfill a course requirement, (b) ninety of the one hundred twenty- 
one Ss were female, and (c) the program of instruction was relatively 
short (215 frames) . 

The finding of this research did not support the major hypo- 
theses, but a number of interesting findings did emerge. There is no 
doubt that the Ss learned during the experimental treatment as shown 
by the significant difference between pretest and posttest scores 
(Test 1 scores) . 

An interesting finding was that the tutor-role Ss showed 
lower performance when compared to the individuals. However, tutor- 
role Ss did not differ from the pupil-role Ss, and pupil-role Ss did 
not differ significantly from the individuals for overall Test 1 
performance. One must remember that Test 1 was based upon the 
material taught in the conventional linear program style (Gibson, 1967), 
and this material, for the most part, was the low difficulty material. 
This finding was also found by Myers, Travers, and Sanford (1965) 
for young children. In their study Ss who gave feedback to others 
showed lower performance than the £s who received verbal feedback. 
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These results may possibly be explained on the basis of: a 
number of tutor verbal protocols. At the conclusion of the 
experiment a number of tutor-role Ss indicated that they misunderstood 
their role-function in that they did not think they would be examined 
on the material, and consequently did not always read the frame along 
with their pupil. Also a few tutors indicated that upon occasion 
they did not have enough time to read the entire frame before, their 
pupil had responded. These types of problems may account of the 
differences observed in the overall Test 1 analysis. 

This finding is not consistent with the test 2 results 
("integration" frames), which showed no significant differences between 
student roles. For the difficult material, tutors do not seem to add 
co the learning process. On the other hand, tutors learned this 
difficult material as well as the other student role-types, even with 
their increased responsibilities. 

The definition of a tutor in this study is quite unlike that 
which is found in the typical school setting. A tutor is usually a 
subject matter expert and adept at explaining student misconceptions. 
Since the "integration" frames were designed to be extremely diffi- 
cult for this population of students, both student tutors and pupils 
probably did not understand the materials. The combination of tutor 
plus explanation, for the above reason, undoubtedly did not contri- 
bute to the learning process. 

It must be remembered that explanation was only found to have 
an effect for Version 3 of the treatment materials (massed "inte- 
gration" frame condition) as seen by the second-order interaction 
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between type of feedback for incorrect answers and type of materials 
(Figure 3). One might hypothesize that this effect was basically 
a recency effect. If this were the case we would have expected a 
decrease in the number of errors of the KR groups on Version 3 as 
well; as the time between learning and testing was the same for both 
groups. The results of a Duncan's Multiple Range Test showed no mean 
differences for Test 2 scores between the KR groups for the three 
versions of the materials. It does seem that explanation did increase 
the learning of the difficult material and that this effect is not just 
a recency factor. 

The results concerning the effects of explanation must always 
be examined with two factors involved: (1) the quality of the 

explanations employed and (2) the extreme "integration" frame 
difficulty level. It is possible that a more profound effect of 
explanation would occur with items of a more moderate difficulty 
level . 

While it was quite clear from this study that the spacing or 
massing of errors did not adversely affect student attitudes as 
hypothesized, the magnitude of positive attitudes appears to be 
dependent upon the number of errors committed upon the test instru- 
ments, at least for the paired groups. This was found for PI 
attitudes and the Computer-Based Testing Attitudes. The effect is 

t 

consistent for High and Low attitudes tutors and pupils, but not 
as consistent for individuals. Some individuals performed relatively 
well and consequently put forth favorable attitudes, while other 
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individuals showed good performance and disliked the experience re- 
gardless of the treatment materials studied. 

/ 

Initial student attitudes toward modern technology in 
general were not found to significantly effect performance. High 
"orientation" attitude Ss did not perform significantly better than 
low "orientation" attitude j3s . This finding is consistent with the 
results of Wodtke (1965), Eigen and Feldhusen (1964), and Mathis, 
Smith, and Hansen (1968). The investigators found that student 
attitudes depend upon immediate experienced factors such as number of 
errors committed and familiarity with the subject matter. The 
observed Computer-Based Testing attitudes reflect this relationship 

very clearly with the exception of the Low attitude individuals who 
performed well. 

An interesting finding concerned the use of confidence ratings 
in the determination of internal consistency reliability of the Test 1 
data. The use of the product of "correctness" (+1 or -l)times the. 
confidence rating increased Test 1 reliability from .645 to .687. 

This is not a large increase, but lends support to the Massengill and 
Schuford (1967) notion that confidence ratings add more information 
concerning test responses than can be obtained from the correct or 
incorrect responses themselves. 
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APPENDIX A 



DIRECTIONS FOR PRE-TEST, PART 1 

The following ten-item test is taken from the program unit on 
intelligence testing which you will be studying from on Tuesday. These 
items are typical of the concepts involved in the unit. 

Answer each item as best you can. Please answer all ten items 
on the separate answer sheet . Please make sure to print your name 
clearly. 

1. Which test would it be best to administer to a nine-year-old Greek 
boy who has been in American three months? 

a. Stanford-Binet Intelligent"' Scale 

b. Kuhlmann-Anderson Intelligence Scale 

c. Arthur Point Scale 

d. Otis Quick-Scoring Mental Ability Test 

2. A child whose family often play number games with him is able to 
increase his score on the numerical reasoning portion of an IQ 
test. This indicates that: 

a. the child's innate capacity cannot be increased by 
experience . 

b. the child's ability to answer test questions may be in- 
creased by experience. 

c. the child will also be able to increase his score on verbal 
reasoning since numerical and verbal abilities are 
moderately correlated. 

d. b and c only are correct. 

3 . The amount of score fluctuation to be expected in most cases 
when a second form of a test is given is measured by the: 

a. standard error 

b. standard deviation 

c. validity 

d . mental age 

4. A WAT.S IQ score of 100 is obtained by every person; 

a. answering correctly every item on the test. 

b. answering correctly 100 items on the test. 

c. who has a MA equal to his CA. 

d. obtaining the mean score obtained by a standardization 
sample, of his own CA. 

5. To establish the reliability of a test: 

a. the test items should be carefully examined for objectivity. 

b. one needs to know the extent to which it is valid. 

c. it must be administered to the same group of people twice, 

d. one needs an outside criterion of some sort. 
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6. The Wechsler tests of intelligence: 

a. have different test items for each age level, 

b. have a verbal, performance, and full-scale IQ. 

c. can be administered to groups. 

d. have standard deviations of 20 IQ points 

7. Which of the following is likely to increase the potential capacity 
of the individual? 

a. favorable environmental conditions 

b. stimulating educational experiences 

c. excellent guidance services 

d . none of the above 

e . a and b only 

8. Miss Henderson taught in a large urban high school last year where 
the students were, drawn from various sections of the city. Next 
year she will teach in a homogeneous group of students. She can 
expect what changes between the relationship of IQ and school 
achievement? 

a. it will increase 

b. it will decrease 

c . there should be no change 

d. cannot tell 

9. Miss Cole was asked to make an assessment of a student in her 
class who was doing poorly. The intelligence test she should most 
likely use would be which of the following? 

a. individual test 

* b. group test 

c. test battery* 

d. none of the. above 

10. An anxious mother was insistent upon determing whether or not her 
small son possessed the special aptitude for creative writing that 
her famous husband has. To arrive at the most satisfactory con- 
clusion, which of the. following tests could be used? 

a. WAIS 

b. Multi-test Battery 

c. Stanfora-Bine.t 

d . SAT 

e. Kuhlmann -Anders on 



DIRECTIONS FOR PRE-TEST, PART 2 

The following ten- item test is taken from the program unit on 
Piaget s theory on inrellectual functioning which you will be studying 

in a few minutes. These items are typical of the concepts involved in 
the unit. 
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Answer each item as best you can. Please answer all ten items 
on the answer sheet provided. Please make sure to print your name 
and code number clearly. 



1. A child, who uses a stool to reach a cookie jar, one day cannot 
find the stool and uses a box. In Piaget’s theory he is: 

a. accommodating 

b. assimilating 

c. showing evidence of formal operations 

d. showing a circular reaction 

2. The order in which stages of mental development occur: 

a. appears to be constant from culture to culture. 

b. appears to differ from child to child 

c. appears to differ from environment to environment 

d. is highly related to the background of the learner 

3. A ’’normal” rate of mental development: 

a. requires adequate stimulation 

b. is dependent solely on maturation 

c. can occur without stimulation in early stages of 
development 

d. always results from children ’’normal” at birth 

4. Verification of Piaget’s findings are difficult to obtain because; 

a. the backgrounds of children he worked with are different 
from that of many other groups of children 

b. the particular developmental stages enumerated may be a 
result of che kinds of problems used in testing 

c. both of the above are correct 

d. neither of the above are correct 



5. 



The situation in which the child shows new behaviors or modifica- 



tions of past behaviors is called 

a. assimilation 

b. accommodation 

c. matching 

d. resemblance 



by Piaget. 



6. Preliminary grouping of equalities (i.e., if A=B and B=C, then 
A=C) are, according to Piaget, most likely to occur in; 

a. preoperational thought stage 

b. formal operations stage 

c. concrete operations stage 

d. mental combinations stage 

7. The major characteristic of the period of formal operations is* 

a. engaging in hypothetico-deductive thinking 

b. capacity for propositional thinking 

c. thinking which involves combinatorial analysis 

d. seeing the possible in the real 
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8. The stage in which the child is perceptually dominated by the 
stimulus is the: 

a. circular reaction stage 

b. intuitive stage 

c. concrete stage 

d. preconceptual thought stage 

9. Piaget distinguishes between what two major stages of mental 
development? 

a. sensorimotor — formal 

b. concrete-formal 

c. sensorimotor — conceptual 

d. concrete operation stage 

10. What stage of mental development occurs during the years four 
to seven? 

a. stage of preconceptual thought 

b. intuitive stage 

c. circular reaction stage 

d. concrete operation stage 
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ATTITUDE SCALE TOWARD INSTRUCTIONAL MEDIA 

This is not a test of information; therefore, there is no one 
"right" answer to a question. We are interested in your opinion on 
each of the statem'” 3 below. Your opinions will be strictly confi- 
dential. Do not hesitate to put down generally how you feel about 
each item. We are seeking information; not compliments. 



Do not spend too much time on any one statement, but give the 



answer which seems to describe how you generally feel. 

CIRCLE THE RESPONSE THAT MOST NEARLY REPRESENTS YOUR REACTION TO 
EACH OF THE STATEMENTS BELOW: 

1. New approaches to education (i.e., language labs, film strips, 
slides, computer-assisted instruction, etc.) usually challenge 
me to do my best work. 

Strongly Disagree Uncertain Agree Strongly 

disagree agree 

2. New technological innovations in education stir my interest in 
trying to find out more about the subject matter. 



3. Learning becomes too mechanical when studying via the new 
approaches to education. 



4. Technological innovations in education are an inefficient use of 
the student's time. 



Strongly Disagree 
disagree 



Uncertain Agree Strongly 

agree 



Strongly Agree 
agree 



Uncertain Disagree Strongly 

disagree 



Strongly Disagree Uncertain Agree 
disagree 



Strongly 

agree 



5. New approaches to education are inflexible. 



Strongly Agree 
agree 



Uncertain Disagree Strongly 

disagree 
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Even otherwise interesting material would be boring when presented 
by new techniques in education. 



Strongly Agree Uncertain Disagree Strongly 

a ® ree disagree 

I am not in favor of technological innovations in education because 
they are just another step toward de-personalized instruction. 

Strongly Disagree Uncertain Agree Strongly 

disagree agree 

I would say that the new techniques in education are superior to 
the tranditional methods of instruction. 



Strongly Agree 
agree 



Uncertain Disagree Strongly 

disagree 



I will be concerned that I might not be understanding the material. 



Strongly Agree 
agree 



Uncertain Disagree Strongly 

disagree 



In a situation where I am trying to learn something, it is 
important to me to know where I stand relative to others. 



Quite 

often 



Often 



Occasionally Seldom 



Very 

Seldom 



APPENDIX C 



TEST 1 



1. Which test would it be. best to administer to a nine-year-old Greek 
boy who has been in American three months? 

a. Stanford-Binet Intelligence Scale 

b. Kuhlmann-Anderson intelligence Test 

c. Arthur Point Scale 

d. Otis Quick-Scoring Mental Ability Test 

2. Which test would probably give the highest IQ to a seventy-three- 
year-old woman who reads z lot? 

a. 1937 Stanford-Binet 

b . WAIS 

c . WISC 

d. Arthur Point Scale 

3. A child whose family often play number games with him is able to 
increase his score on the numerical reasoning portion of an IQ 
test. This indicates that: 

a. the child's innate capacity cannot be increased by experience. 

b. the child's ability to answer test questions may be increased 
by experience, 

c. the child will also be able to increase his score on verbal 
reasoning since numerical and verbal abilities are moderately 
correlated . 

d . b and c only are correct 

4. The amount of score fluctuation to be expected in most cases 
when a second form of a test is given is measured by the: 

a. standard error 

b. standard deviation 

c. validity 

d . mental age 

5. In order to adequately interpret an IQ score, we need to know: 

a. What abilities the IQ test is supposed to measure. 

b. whether the test is verbally-oriented or performance-oriented. 

c. the educational background and physical background of the 
subject. 

d. all of these 

6. Perfect correlation between two IQ tests indicates that; 

a. both tests are valid. 

b. both tests are good measures of IQ. 

c. the scores on the first test always "go" with scores on the 
second. 

d. the two tests are identical in format. 
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7. A WAIS IQ score of 100 is obtained by every person: 

a. answering correctly every item on the test. 

b. answering correctly 100 items on the test. 

c. who has an MA equal to his CA 

d. obtaining the fnean score obtained by a standardization 
sample of his own CA 

8. The best test of intelligence for a seemingly bright mute child 
of six would be: 

a. Stanford-Binet 

b. Arthur Point Scale 

c . WAIS 

d . WISC 

9. A valid test is one that: 

a. is highly correlated with an accepted criterion measure. 

b. shows a high correlation between socres obtained on two 
forms of the same test. 

c. is acceptable at the 100 per cent criterion level. 

d. uses no essay questions. 

10. Which type of instrument should not be administered by a teacher? 

a. standardized achievement 

b. individual intelligence 

c. group intelligence 

d. interest 

11. A child eight years and six months old has a mental age of fourteen 
years and one month. His IQ is: 

a. 100 

b. 102 

c. 128 

d. 146 

e. 166 

12. Occasionally children are assigned to long-term instructional 
programs solely on the basis of an IQ score. This practice is 
largely due to the incorrect assumption that: 

a. IQ alone is an adequate basis for assigning children to 
programs „ 

b. scores on IQ tests reflect accurately an important ability. 

c. IQ tests can be used with comparable validity for all children. 

d. all of the above. 

13. Which of the. following does not belong with the other three? 

a. standardized administration 

b. standardized validity 

c. standardized norirts 

d. standardized scoring 
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14. Pat is a poor reader and achieves an IQ score of 30 on the Otis 
Quick-Scoring lest. We can conclude that; 

a. he is brighter than the test indicates. 

b. his poor reading may lower his performance on the test. 

e. he would probably score higher on a different group IQ test, 
d. he is too dull to profit from school. 

15. To establish the reliability of a test; 

a. the test items should be carefully examined for objectivity. 

b. one needs to know the extent to which it is valid. 

c. it must be administered to the same group of people twice. 

d. one needs an outside criterion of some sort. 

16. The extent to which a test measures what it purports to measure 
constitutes its; 

a. reliability 

b. validity 

c. standard deviation 

d. coefficient of correlation 

17. The Wechsler tests of intelligence: 

a. have different test items for each age level. 

b. have a verbal „ performance, and full-scale IQ. 

c. can be administered to groups 

d. have standard deviations of 20 IQ points. 

18. Most intelligence tests are designed to measure the ability or 
abilities involved in: 

a. abstract thinking 

b. adapting to new situations 

c. creative thinking 

d. numerical reasoning 

19. Which of the following is likely to increase the potential capacity 
of the individual? 

a. favorable environmental conditions 

b. stimulating educational experiences 

c. excellent guidance services 

d. none of the above 

e. a and b only 

20. Miss Henderson taught in a large urban high school last year where 
the students were drawn from various sections of the city. Next 
year she. will teach in a homogeneous group of students. She can 
expect what changes between the relationship of IQ and school 
achievement? 

a. it will increase. 

b. it will decrease, 

c. there should be no change. 

d. cannot tell. 
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21. Miss Cole was asked to make an assessment of a student in her 
class who was doing poorly. The intelligence test she should 
most likely use would be which of the following? 

a. individual test 

b. group test 

c . test battery 

d. none of the above 

22. Intelligence as potential capacity is probably a function of: 

a. heredity 

b. congenital development 

c . growth 

d. all of the above 

e. a and b only 

23. If there is a correlation coefficient of -.02 between physical 
education and reading scores, one might conclude: 

a. a good reader will be good in physical education. 

b. a poor reader will be good in physical education. 

c. there, is little relation between these variables. 

d. none of the above 

24. The relative mental -ability standing many students decreases 

as they proceed from grade school to high school to college be- 
cause: 

a. their mental abilities decrease. 

b. they become less able to learn. 

c. mental growth stops in the early twenties. 

d. the average mental abilities of students are higher in college. 

25. An anxious mother was insistent upon determining whether or not 
her small son possessed the special aptitude for creative writing 
that her famous husband has. To arrive at the most satisfactory 
conclusion, which of the following tests could be used? 

a. WAIS 

b. Multi-test Battery 

c . Stanf ord-Binet 

d. SAT 

e . Kuh lmann-Ande r s on 

26. A child, who uses a stool to reach the cookie jar, one day dan't 
find the stool and uses a box. In Piaget’s theory he is; 

a. accommodating 

b. assimilating 

c. showing evidence of formal operations 

d. showing a circular reaction 

27. According *"o Piaget, cognitive development is due to; 

a. maturation 

b. interaction with the environment 
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c. a and b functioning together 

d . none of these 

28. Piaget's concept of accommodation refers to the child's tendency to: 

a. relate what new stimuli he perceives to what he already knows 
how to do. 

b. utilized less information in making a perceptual recognition 

c. change his older conceptual understandings so that they fit 
new perceptions. 

d. telescope two concepts into a superordinate concept. 

29. According to Piaget's theory, there should always b e 
discrepancy between the level of difficulty of new problems pre- 
sented in a learning situation and the child's level of mental 
development . 

a. a small 

b. a large 

c . no 

d. some optima?, amount of 

30. The order in which stages of mental development occur: 

a. appears to be constant from culture to culture. 

b. appears to differ from child to child. 

c. appears to differ from environment to environment. 

d. is highly related to the background of the learner. 

31. When children reach the stage of , they are ready to 

learn totally abstract concepts. 

a. preconceptual thought 

b. formal operations 

c. concrete operations 

d. sensorimotor development 

32. In his studies, Piaget used chidren; 

a. from upper-socioeconomic classes. 

b. from middle-class backgrounds with "average" am ounts of 
stimulation. 

c. with exceptionally high IQ scores. 

d. with exceptionally low IQ 

33. We know from Piaget's research that his nomothethic laws appear 
to hold true for the development of the concepts of: 

a. logic, mathematics, and physical phenomena. 

b. biological and social phenomena. 

c. artistic and dramatic phenomena. 

d. all of these 

34. A "normal!' rate of u-eital development: 

a. requires adequate stimulation 

b. is dependent solely on maturation. 




c. . can occur without, stimulation in early stages of development. 

d. always results from children "normal" at birth. 

Verification of. Piaget's findings are difficult to obtain because 

a. the backgrounds of children he worked with are different 
from that of many other groups of children. 

b. the particular developmental stages enumerated may be a 
result if the kinds of problems used in testing 

c. . both of the above are correct 

d. neither of the above are correct. 

Piaget gathered his data by which one of the following methods? 

a. experimental method 

b. logitudinal method 

c. . observational method 

d. analytical method 

The situation in which the child shows new behaviors or 
modifications of past behaviors is called bv Piaeet. 

a. assimilation 

b . accommodat i on 

c. matching 

d. resemblance 

Intelligence test theory and Piaget's theory are similar in which 
of the following ways? 

a. both are concerned with similar intellectual tasks 

b. both are concerned with hereditary characteristics 

c. both contain a time dimension 

d. both a and c are correct 

A child in the period of concrete operations of thinking can; 

a. understand what is real 

b. distinguish between the possible and the real 

c. . see how the real can lead to the possible 

d. consider the possible, in abstract thought 

Helen, a 1.6 year-old Negro attended a predominately white high 
school. When she first enrolled she had few friends but became 
we 11 -acquainted after the first year, primarily because she 
changed her conception of white students. This is an example of; 

a. accommodation 

b. assimilation 

c. organization 

d. none of the above 
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41. Preliminary grouping of equalities (i.e., if A^B and B=G, then 
A=C) are, according to Piaget, most likely to occur in; 

a. preoperational thought stage 

b. formal operations stage 

c. concrete operations stage 

d. mental combinations stage 

42. The major characteristic of the period of formal operations is: 

a. engaging in hypothetico-deductive thinking 

b. capacity for propositional thinking 

c. thinking which involves combinatorial analysis 

d. seeing the possible in the real 

43. The situation in which the child learns to make new responses to 

familiar stimuli is called by Piaget. 

a. assimilation 

b. accommodation 

c . matching 

d. resemblance 

44. The final stage of the sensorimotor period is the: 

a. reflexive stage 

b. mental combinations stage 

c. circular reaction stage 

d. preconceptual thought stage 

45. Assimilation often implies which of the following? 

a. learning 

b. generalization 

c. discrimination 

d. a and b only 

e. all of the above. 

46. The stage in which the child is perceptually dominated by the 
stimulus is the: 

a. circular reaction stage 

b. intuitive stage 

c. concrete stage 

d. preconce.ptual though stage 

47. Accommodation refers to: 

a. an internal change, in a persons 

b. a change in an external idea 

c. the functional invariant of organization 

d. the assimilation of new ideas 
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48. Piaget distinguishes between what two major stages of mental 
development? 

a. sensorimotor -formal 

b. concrete -formal 

c. sensorimotor-conceptual 

d. intuitive-conceptual 

49. What stage of mental development occurs during the years 4*7? 

a. stage of preconceptual though 

b. intuitive stage 

c. circular reaction stage 

d. concrete operation stage 

50. The reflexive stage takes place during what age range? 

a. the first month of life 

b. from the first to the eighth month 

c. from the eighth month to the eighteenth month 

d. beyond the eighteenth month. 




APPENDIX D 



STUDENT ATTITUDE TOWARD PROGRAMMED INSTRUCTION 
COMMUNICATION SKILL 



This is not a test of information; therefore, there it no one 
"right" answer to a question. We are interested in your opinion on 
each of the statements below. Your opinions will be strictly confi- 
dential. Do not hestitate to put down exactly how you feel about each 
item. We are seeking information; not compliments. Please be frank. 

Name : Date : 

Name of Course: 



CIRCLE THE RESPONSE THAT MOST NEARLY REPRESENTS YOUR REACTION TO EACH 
OF THE STATEMENTS BELOW: 



While taking programmed instruction, I felt challenged to do my 
best work. 



Strongly 

disagree 



Disagree 



Uncertain 



Agree 



Strongly 

agree 



I was concerned that I might not be understanding the material. 

5 : : : 

Disagree Uncertain Agree 



Strongly 

disagree 



Strongly 

agree 



3. I was not concerned when I missed a question because no one was 
really watching me carefully. 



Strongly 

disagree 



Disagree 



Uncertain 



Agree 



Strongly 

agree 



4. While taking programmed instruction, I felt isolated and alone. 



All of 
the time 



Most of 
the time 



Some of 
the time 



Only Never 
occasionally 



I felt uncertain as to my performance in the programmed course 
relative to the performance of others. 



All of 
the time 



Most of 
the time 



Some of Occasionally 
the time 



Never 
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6. I found myself just trying to get through the material rather than 
trying to learn. 

• • • i : 

All of Most of Some of Occasionally Never 

the time the time the time 



7 . I knew whether my answer was correct or not before I was told . 



Quite Often Occasionally Seldom Very 

°ft© n seldom 



8. In a situation where I am trying to learn something, it is important 

to me to know where I stand relative to others. 

: : • • . 

• • • • 

Quite Often Occasionally Seldom Very 

seldom 



9. As a result of having studied some material by programmed 

instruction, I am interested in trying to find out more about the 
subject matter. 

: : : : 
Strongly Disagree Uncertain Agree Strongly 

disagree agree 

10. I was more involved in verbal interaction with my partner than in 
understanding the material. 

• i : : : 

All of Most of Some of Only Never 

the time the time the time occasionally 






11. 


I felt I 
• 


could work at my 
• 


own pace with programmed 

A ^ 


instruction. 




• 

Strongly 

disagree 


• 

Disagree 


• 

Uncertain 


• 

Agree 


• 

• 

Strongly 

agree 


12. 


Programmed instruction makes the learning 
• • * 


too mechanical. 




• 

Strongly 

disagree 


• 

Disagree 


• 

Uncertain 


• 

Agree 


• 

• 

Strongly 

agree 


13. 


I felt as 
• 


if I had a private tutor while taking the 
• * 


program. 




• 

Strongly 

disagree 


• 

Disagree 


• 

Uncertain 


• 

Agree 


• 

• 

Strongly 

agree 



14. I was aware of efforts to suit the material specifically to me. 



Strongly Disagree 

disagree 



Agree Strongly 

agree 




Uncertain 



I found it difficult to concentrate on the course material 

because of the (teacher or pupil) . 

• • • • . 

• • • • : 

All of Most of Some of Only Never 

the time the time the time occasionally 



Questions were asked which I felt were not relevant to the 
materials presented. 

: : : 

All of Most of Some of Only Never 

the time the time the time occasionally 



Programmed instruction is an inefficient use of the student's time. 

: ; . 

1 • • • 

Strongly Disagree Uncertain Agree Strongly 

agree agree 



Programmed instruction made it possible for me to learn quickly. 



Strongly 

disagree 



Disagree 



Uncertain 



Agree 



Strongly 

agree 



I felt frustrated by the programmed instruction situation. 

: : 

Disagree Uncertain Agree 



Strongly 

disagree 



The programmed instruction approach is inflexible. 
: : : 

Strongly Disagree Uncertain Agree 

disagree 



Strongly 

Agree 



Strongly 

agree 



Even otherwise interesting material would be boring when presented 
by programmed instruction. 



Strongly 

disagree 



Disagree 



Uncertain 



Agree 



Strongly 

agree 



In view of the effort I put into it, I was satisfied with what I 
learned while taking programmed instruction. 



• • • 
Strongly Disagree Uncertain Agree 

disagree 



Strongly 

agree 



In view of the amount I learned, I would say that programmed 
instruction is superior to any other instruction. 



Strongly 

disagree 



Disagree 



Uncertain 



Agree 



Strongly 

agree 
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24. With a course such as I took by programmed instruction, I would 
prefer programmed instruction to any other instruction. 





Strongly 

disagree 


Disagree 


Uncertain 


Agree 


Strongly 

agree 


25. 


1 am not in 

another step 
• 


favor of programmed instruction because it 
1 toward de-personalized instruction. 


is just 




• 

Strongly 

disagree 


• 

Disagree 


• 

Uncertain 


• 

• 

Agree 


• 

• 

Strongly 

agree 


26. 


Programmed instruction is 

• • 


too fast. 








• 

Strongly 

disagree 


• 

Disagree 


• 

Uncertain 


• 

• 

Agree 


• 

• 

Strongly 

agree 


27. 


Programmed 


instruction is 


boring. 








Strongly 

disagree 


Disagree 


Uncertain 


Agree 


Strongly 

agree 



APPENDIX E 



STUDENT ATTITUDE TOWARD COMPUTER-BASED TESTING 



This is not a test of information; therefore, there is no one "right' 1 
answer to a question. We are interested in your opinion on each of the 
statements below. Your opinions will be strictly confidential. Do not 
hesitate to put down exactly how you feel about each item. We are seek- 
ing informaion; not compliments. Please be frank. 

Name : Date : 

Name of Course 



CIRCLE THE RESPONSE THAT MOST NEARLY REPRESENTS YOUR REACTION TO EACH 
OF THE STATEMENTS BELOW: 



1 . 



2 . 



3 . 



4 . 



5 . 



6 . 



While taking the computer test, I felt challenged to do my best. 



Strongly 

disagree 



Disagree 



Uncertain 



Agree 



Strongly 

agree 



I was concerned that I might not understand the material. 



Strongly 

disagree 



Disagree 



Uncertain 



Agree 



Strongly 

agree 



While taking the computer test, I felt isolated and alone. 



All of 
the time 



Most of 
the time 



Some of 
the time 



Only Never 

occasionally 



I guessed at the answers to questions. 



Quite 

often 



Often 



Occasionally 



Seldom 



Very seldom 



I was more involved in running the machine than in understanding 
the question. 



All of 
the time 



Most of 
the time 



Some of 
the time 



Only Never 

occasionally 



I was aware of efforts to suit the material specifically to me 



Strongly 

agree 



Agree 



Uncertain 



Disagree 



Strongly 

disagree 
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7. The computer situation made me feel quite tense. 

Strongly Disagree Uncertain Agree 

disagree 

8. Questions were asked which I felt were not relevant. 

All of Most of Some of Only 

the time the time the time occasionally 

9. I could have done better if I hadn’t felt pushed. 

Strongly Agree Uncertain Disagree 

agree 

10. I would say computer testing is superior to class testing 

Strongly Disagree Uncertain Agree 

disagree 



Strongly 

agree 



Never 



Strongly 

disagree 



Strongly 

Agree 



APPENDIX F 



STUDENT DIRECTIONS FOR TUTORS IN THE EXPLANATION CONDITIONS 

The primary purpose of the introductory course in educational 
psychology is just that — to introduce the student to educational 
psychology. In the past this has been attempted via the usual reading 
assignments, class lectures, and periodic examinations. The usual 
results were that most students seemed to learn an adequate amount of 
material, but many did not. 

This term we shall attempt to improve the rate of student 
learning via the use of new technological educational methods and 
equipment, i.e. programmed instruction. You have been selected to 
participate in an experimental learning experience. 

DUTIES OF THE "INSTRUCTOR." 

Your role is that of an "Instructor ." You will have a copy of 
an "instructor’s" manual containing program frame answers and comments 
to make (sic) to assist ”your" student in answering the study materials. 
If the student should give a correct answer, you should tell him so by 
saying, "That is right, good, correct,” or a similar comment. Do not 
be afraid to encourage the student to answer the question. In some 
instances, your "student” may have a great deal of difficulty in 
formulating any kind of answer. At such points, the ’’instructor” 
should make every effort to get some sort of response from the ’’student.” 
It isn’t at all necessary that the ’’student” always give a brilliant 
answer (or even a correct answer) . The real point is to get both 
’’instructor” and ’’student” actually thinking about the question. 
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Please do not just dismiss all questions with a "who knows?" attitude. 

There are, for the most part, questions that one might reasonably 
expect college students to occasionally ponder. If both "instructor" 
and "student" do in fact give these questions serious thought., a good 
answer will almost always be the result. One of the surest signs of 
intellectual maturity is the tendency to think seriously about serious 
questions and formulate some sort of answer. (On the other hand, there 
is no point in prolonging a discussion beyond the point chat it seems 
mutually profitable.) 

The goal here is to understand and discuss the material when 
problems arise, not to get through it as quickly as possible. If you 
are unlucky enough to draw a "student" who gives only a "yes" or "no"-- 
or "beats me"--kind of answer to a difficult question, accept this as 
a challenge! Try to modify his behavior via the techniques you have 
learned from your education courses . If you do go skipping through 
the program hurriedly, the whole bit will be a loss mostly to you and 
the "student." 

SPECIFIC DIRECTIONS 

1. Your student should read each frame silently and then 
orally give you his answer. 

2. If he is correct — tell him so and continue to the. next frame. 

3. If he makes an error, use the comments provided as guidelines 
in helping to understand his errors. 

4. Write down his response as best you can in your answer 
booklet. In some cases, only a letter or number answer is required. 
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Use the guidelines provided . If you are cold to review 
certain frames, review them . In ocher cases you are left: to your own 
ingenuity. Use it . 

6. Remember — your goal as an "instructor" is to have your 
student learn the material. Feel fre,?. to discuss any errors or pro~ 

bleras and to insert your own comments wherever you think they will be 
mo5t beneficial to your student. 

7. Your job is a tough one. Accept the challenge and do the 
best job you possibly can. Your "instructor" position will be the same 

when you acquire a teaching job in a school sys>tem»“to teach , Do just 
that. 

If at any point you or your student do not understand the 
Program, consult one of the "assistants" on duty. 

Please keep track of all £ime spent on each unit in minutes. 

If you and your student spend an hour and a half on Unit 1 5 place the 
number 90 on the front page of your manual. This is important. 

your experience this quarter seems to warrant continuing, 

we shall make every effort to develope this type of material for further 
student use. 



STUDENT DIRECTI ONS FOR PUPILS IN THE EXPLANATION OONDIITCNS 

The primary purpose of the introductory course in educational 

psychology is just that—to introduce the student to educational 

psychology In the past this has been attempted via the usual reading 

assignments, class lectures, and periodic examinations. The usual 

results were that most: students seemed to learn an adequate amount of 
material, but many did not. 
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This term we shall attempt to improve the rate of student 
learning via the use of new technological educational methods and 
equipment, i.e., programmed instruction . You have been selected to 
participate in an experimental learning experience. 

DUTIES OF THE “STUDENT " 

Your job is that of a "student." Basically the "studoiit" 
needs only to follow the directions of his I'instructor ." 

THE MOST IMPORTANT DUTY OF THE "STUDENT" IS TO MAKE A VERY 
CONSCIENTIOUS EFFORT TO ANSWER ALL QUESTIONS TO THE BEST OF HIS ABILITY . 
Even if you have absolutely no idea what the "correct" answer might be, 
give the matter some serious though and then indicate, your best guess. 

At the very least, do not discourage your "instructor" by showing any 
indication that you are incapable of even thinking about the question. 

SPECIFIC DUTIES 

1. You are to read each frame silently. When you have for- 
mulated an answer, tell your "instructor"your answer. 

2. Your "instructor" will write your answer down and then 
tell you if it is correct or not. 

3. If you are incorrect, your "instructor" will try to assist 
you in understanding what the correct answer is, 

4. Do not be afraid to verbally interact with your "instruc- 
tor." He is trying to do his best to help you get the most out of the 
materials. To help him you should do the best job you can in answering 
the program frame questions. 
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5. Feel free to discuss any questions you have with your 
"instructor 

If at any point you or your "instructor" do not understand 
the program , consult one of the "assistants" on duty. 

If your experience this quarter seems to warrant continuing, 
we shall make every effort to develop this type of material for further 
student use. 



STUDENT DIRECTIONS FOR TUTORS IN THE KNOWLEDGE 
OF RESULTS CONDITIONS 

The primary purpose of the introductory course in educational 
psychology is just that — to introduce the student to educational 
psychology. In the past this has been attempted via the usual reading 
assignments, class lectures and periodic examinations. The usual 
results were that most students seemed to learn an adequate amount of 
material, but many did not. 

This term we shall attempt to improve the rate of student 
learning via the use of new technological educational methods and 
equipment, i.e., programmed instruction. You have been selected to 
participate in an experimental learning experience. 

DUTIES OF "INSTRUCTOR " 

Your role is that of an "instructor." You will have a copy 
of the program unit and answer booklet. Only your booklet will contain 
answers to the questions. 

After securing the booklets from the "assistants," you will 
direct your Vstudent" to read silently each program frame. After he 

ERIC 
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has finished reading it, he will orally give you his answer. Write 
that answer down on the answer sheet. 

If your student is correct, tell him so by saying, "That's 
right, good, correct," etc. If he is wrong or partially wrong, tell 
him that he is wrong and give him the correct answer. 

The correct answer completion should be given only after it 
is quite apparent that your "student" is not likely to ever get the 
correct answer short of being supplied with it directly. If the 
"student" makes no effort to provide the correct completion for the 
frame, encourage him to give some kind of answer. 

Your major function as an "instructor" is to ensure that the 
student proceeds in a conscientious fashion and to give him the correct 
answers when he fails to give the correct response. Following this 
method, both "instructor" and "student" should learn a great deal. 

If your experience this quarter seems to warrant continuing, 
we shall make every effort to develop this type of material for further 
student use. 

If at any point, you or your "student" do not understand the 
program , consult one of the assistants on duty. 

Keep an accurate record of amount of time (in minutes) spent 
on both program units. Indicate this on the cover of your teacher's 




manual . 
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STUDENT DIRECTIONS FOR PUPILS IN THE 
KNOWLEDGE OF RESULTS CONDITIONS 

The primary purpose of the introductory course in educational 
psychology is just that — to introduce the student to educational 
psychology. In the past this has been attempted via the usual reading 
assignments, class lectures, and periodic examinations. The usual 
results were that most students seemed to learn an adequate amount of 
material, but many did not. 

This term we shall attempt to improve the rate of student 
learning via the use of new technological educational methods and 
equipment, i.e., programmed instruction. You have been selected to 
participate in an experimental learning experience. 

DUTIES OF "STUDENT" 

Your job is that of a "student." Basically the "student" 
needs only to follow the directions of his "instructor." 

THE MOST IMPORTANT DUTY OF THE "STUDENT" IS TO MAKE A VERY 
CONSCIENTIOUS EFFORT TO ANSWER ALL QUESTIONS TO THE BEST OF HIS ABILITY . 

Even if you have absolutely no idea what the "correct" answer 
might be, give the matter some serious thought and then indicate your 
best guess. At the very least, do not discourage your "instructor" 
by showing any indication that you are incapable of even thinking 
about the question. 

SPECIFIC DUTIES 

1. Read each frame silently. When you have formulated an 
answer, tell your "instructor" your answer. 
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2. Your "instructor" will write, your answer down and then tell 
you if it is correct or not. 

3. If you are incorrect, your "instructor" will give you the 
correct answer. 

It is very important that you not look ahead in the j>rogram! 

To "answer" a question by looking at the next frame will defeat most of 
the design of the project. It is not so important that you always get 
the "right answer" as than you follow very precisely the instructions 
of the program and your "instructor." 

STUDENT DIRECTIONS FOR INDIVIDUALS IN 
THE EXPLANATION CONDITIONS 

The primary purpose of the introductory course in educational 
psychology is just that— to introduce the student to educational 
psychology. In the past this has been attempted via the usual reading 
assignments, class lectures, and periodic, examinations. The usual 
results were that most students seemed to learn an adequate amount of 
material, but many did not. 

This term we shall attempt to improve the rate of student 
learning via the use of new technological educational materials and 
equipment, i.e., programmed instruction. You have been selected to 
participate in an experimental learning experience. 

DUTIES 

Your job is that of a student working his way through the 
programmed units individually at your own rate. 
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THE HOST IMPORTANT DUTY IS TO MAKE A VERY CONSCIENTIOUS 
EFFORT TO ANSWER ALL QUESTIONS TO THE BEST OF YOUR ABILITY . 

Even if you have absolutely no idea what the "correct" answer 
might be, give the matter some serious thought and then indicate your 
best guess. 

SPECIFIC DUTIES 

1. Read each frame carefully. When you have formulated an 
answer, write your answer in the program unit itself where indicated. 

2. Then check your answer with the one provided in your answer 

booklet. 

3. If you were correct, continue on to the next frame. 

4. If you were incorrect, circle your answer and then read the 

comments under the. heading Explanation provided for each frame. 

5. In some cases, a restatement of the question is given as an 

aid to help you in understanding the concepts. In other cases, you 
are told to review previous frames. Follow the instructions carefully 
because they were written with your interest in mind. 

6. Sometimes you are left to your own. ingenuity. USE IT . 
Review or continue as you see best in these cases. 

It is very important that you do not look ahead in the. program ! 
To "answer" a question by looking at. the. next frame will defeat most of 
the design of the. project. It. is not so important that you always get 
the right answer as that, you follow very precisely the instructions 



o 



of the program. 
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On the front page of each unit, keep tract of all the time in 
minutes which you spend on each unit. If you spend 1 hour on Unit 1, 
you should put the number on the front page of Unit 1 ^ and so on. 

If your experience this quarter seems to warrant continuing, 
we shall make every effort to develop this type of material for further 
student use. 



STUDENT DIRECTIONS FOR INDIVIDUALS IN THE 
KNOWLEDGE OF RESULTS CONDITIONS 

The primary purpose of the introductory course in educational 
psychology is just that--to introduce tne student to educational 
psychology. In the past, this has bee - : attempted via the usual reading 
assignments, class lectures, and periodic examinations. The usual 
results were that most students seemed to learn an adequate amount of 
material, but many did not. 

This term we shall attempt to improve the rate of student 
learning via the use of new technological educational methods and 
equipment, i.e., programmed instruction. You have been selected to 
participate in an experimental learning experience. 

DUTIES 

Your job is that of a student working his way through the 
programmed units individually at your own rate. 

THE MOST IMPORTANT DUTY IS TO MAKE A VERY CONSCIENTIOUS EFFORT 
TO ANSWER ALL QUESTIONS TO THE BEST OF YOUR ABILITY . 
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Even if you have absolutely no idea what the "correct" answer 
might be, give the matter some serious thought and then indicate your 
best guess. 

SPECIFIC DUTIES 

1. Read each frame carefully. When you have formulated an 
answer, write your answer in the program unit itself where indicated. 

2. Then check your answer with the one provided in your 
answer booklet. 

3. If you were correct, continue on to the next frame. 

4. If you were incorrect, circle your answer , try to decide 
why you were incorrect when you check the correct answer. 

5 . Continue to the next frame . 

It is very important that you do not look ahead in the program! 
To "answer" a question by looking at the next frame will defeat most 
of the design of the project. It is not so important that you always 
get the right answer as that you follow very precisely the instructions 
of the program. 

On the front page of each unit, keep track of all the time in 
minutes which you spend on each unit. If you spend 1 hour on Unit 1, 
you should put the number 60 on the front page of Unit 1, and so on. 

If your , experience this quarter seems to warrant continuing, 
we shall make every effort to develop this type of material for further 




student use . 
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